---
title: "Belden INFO 3200 Project"
output: 
  flexdashboard::flex_dashboard:
    vertical_layout: scroll
    source_code: embed
---

```{r setup, include=FALSE, warning=FALSE}
#include=FALSE will not include r code in output
#warning=FALSE will remove any warnings from output
library(flexdashboard)
library(tidyverse)
library(GGally)
library(caret) #for logistic regression
library(broom) #for tidy() function
```

```{r load_data}
telcodf <- read_csv("Belden-Telco-Customer-Churnv4.csv")
```

Introduction {data-orientation=rows}
=======================================================================

Row {data-height=250}
-----------------------------------------------------------------------

### Overview 

For this project, we will follow the DCOVAC process. The process is listed below:

DCOVAC – THE DATA MODELING FRAMEWORK

* DEFINE the Problem
* COLLECT the Data from Appropriate Sources
* ORGANIZE the Data Collected
* VISUALIZE the Data by Developing Charts
* ANALYZE the data with Appropriate Statistical Methods
* COMMUNICATE your Results

Row {data-height=650}
-----------------------------------------------------------------------

### The Problem & Data Collection

#### The Problem
The goal of the analysis is to determine which variables are best at predicting whether a customer will “churn”. Churn means the customer stopped using the company's product or service. The dataset is for a telecommunications company called Telco so in the context of the data churn means that the customer terminated their subscription. With this analysis, Telco will have a better idea of what factors contribute to their clients leaving and therefore develop a plan to retain them.

#### The Data
This dataset has 7,044 and 20 variables. For this analysis, we will ignore the `CustomerID` variable which serves as a unique identifier for each customer and does not provide any meaningful information for predicting customer churn.

#### Data Sources
Author(s): Steven Macko
Title: Telco Customer Churn
Year: 2018
Version: 1
Publisher: IBM/Kaggle
URL: https://www.kaggle.com/datasets/blastchar/telco-customer-churn


### The Data
VARIABLES TO PREDICT WITH

* **Gender**: Customer's gender (Male/Female)
* **SeniorCitizen**: Whether the customer is a senior citizen (Yes(0)/No(1))
* **Partner**: Whether the customer has a partner (Yes(0)/No(1))
* **Dependents**: Whether the customer has dependents (Yes(0)/No(1))
* **PhoneService**: Whether the customer has phone service (Yes(0)/No(1))
* **MultipleLines**: Whether the customer has multiple phone lines (Yes(0)/No(1)/No phone service(2))
* **InternetService**: If Customer Has Internet and the Type (DSL/Fiber optic/No(1))
* **OnlineSecurity**: If Customer Has Security on Their Internet (Yes(0)/No(1)/No Internet(2))
* **OnlineBackup**: If Customer Has a Backup for Their Internet (Yes(0)/No(1)/No Internet(2))
* **DeviceProtection**: If Customer has Device Protections Services for Their Internet (Yes(0)/No(1)/No Internet(2))
* **TechSupport**: Whether Customer has Tech Support Enabled on Their Subscription (Yes(0)/No(1)/No Internet(2))
* **StreamingTV**: If the Customer Has StreamingTV Support on Their Subscription (Yes(0)/No(1)/No Internet(2))
* **StreamingMovies**: If the Customer Has Movie Streaming Support on Their Subscription (Yes(0)/No(1)/No Internet(2))
* **Contract**: Type of contract (Month-to-month/One year/Two years)
* **Paperless**: If the Customer Has Opted for Paperless Billing (Yes(0)/No(1))
* **PaymentMethod**: Payment method (Electronic check/Mailed check/Bank transfer/Credit card)
* **MonthlyCharges**: Monthly charges for the customer (continuous variable)
* **TotalCharges**: Total charges accumulated by the customer (continuous variable)

 

VARIABLES WE WANT TO PREDICT

* *Churn*:  Whether the customer churned (Yes(0)/No(1)), Quantitative, response variable)
* *Tenure*: # of Months with Subscription (continuous, response variable)

Data
=======================================================================


Column {data-width=650}
-----------------------------------------------------------------------
### Organize the Data
Organizing data can also include summarizing data values in simple one-way and two-way tables.

```{r, cache=TRUE}
#the cache=TRUE can be removed. This will allow you to rerun your code without it having to run EVERYTHING from scratch every time. If the output seems to not reflect new updates, you can choose Knit, Clear Knitr cache to fix.

#Clean data by replacing spaces with decimals
colnames(telcodf) <- make.names(colnames(telcodf))
#View data
summary(telcodf)
#remove customerID due to it being an identifier
telcodf <- select(telcodf, -customerID)
```
From this data we can see that our variables have a variety of different values and a wide variety of variable types. CustomerID is a unique identifier for each customer and thus serves no purpose in our analysis so we will remove it. Our two dependent variables are Churn and Tenure. Churn is binary with 0 being Yes they did churn and 0 being no they did not churn. Tenure is continuous and we can see that the max tenure is 72 months and the median is 29 months of being subscribed to Telco's service. SeniorCitizen, PhoneService, Partner, and Dependents were recoded from yes/no to 0/1 for easier analysis. Similarly, MultipleLines, OnlineSecurity, OnlineBackup, DeviceProtection, TechSupport, StreamingTV, and StreamingMovies were recoded from Yes/No/No Internet to 0/1/2 for easier analysis. Moreover, MonthlyCharges has a max of 118.75 and a median of 70.35 while TotalCharges has a max of 8684.8 and a median of 1397.5


Column {data-width=350}
-----------------------------------------------------------------------
### Transform Variables
My RStudio was having problems running the code with the variables as factors. Therefore, I had to recode my excel file to replace the characters of these variables with numbers and set them as numeric. 
```{r, cache=TRUE}
telcodf <- mutate(telcodf,
                  SeniorCitizen = as.numeric(SeniorCitizen),
                  Partner = as.numeric(Partner),
                  Dependents = as.numeric(Dependents),
                  MultipleLines = as.numeric(MultipleLines),
                  OnlineSecurity = as.numeric(OnlineSecurity),
                  OnlineBackup = as.numeric(OnlineBackup),
                  DeviceProtection = as.numeric(DeviceProtection),
                  TechSupport = as.numeric(TechSupport),
                  StreamingTV = as.numeric(StreamingTV),
                  StreamingMovies = as.numeric(StreamingMovies),
                  PaperlessBilling = as.numeric(PaperlessBilling),
                  Churn = as.numeric(Churn))

```
#### Customer Churn(Yes(0)/No(1)) & InternetService(DSL,Fiber Optic, No Internet(1))
```{r cache=TRUE}
as_tibble(select(telcodf,Churn) %>%
  table())
as_tibble(select(telcodf,InternetService) %>%
  table())

```
#### Customer Churn (Yes or No)

<!--Instructions to import .jpg or .png images
use getwd() to see current path structure 
copy file into same place as .Rmd file
put the path to this file in the link
format: ![Alt text](book.jpg) -->

![](ChurnDistributionJMP.jpg)


Data Viz #1
=======================================================================


Column {data-width=500}
-----------------------------------------------------------------------
### Response Variables
#### Churn Yes(0)/No(1)
```#{r, cache=TRUE}
#as_tibble(select(telcodf,Churn) %>%
  ##ggplot(aes(y=n,x=Churn)) + geom_bar(stat="identity")
```

We can see we have about 73% of the data as no customer churn and 26.5% that have churned. Looking at the potential predictors related to Customer Churn, we strongest relationships between Tenure, MonthlyCharges, and TotalCharges. The rest of the variable comparisons are further down.


Column {data-width=500}
-----------------------------------------------------------------------

### Transform Variables



```{r, cache=TRUE}
ggpairs(select(telcodf,Churn,tenure,MonthlyCharges,TotalCharges,Contract))
```
Data Viz #2
=======================================================================


Column {data-width=500}
-----------------------------------------------------------------------
### Response Variables

#### Tenure & Churn
```{r, cache=TRUE}
ggplot(telcodf, aes(tenure)) + geom_histogram(bins=60)

ggplot(telcodf, aes(x = Churn)) + geom_bar()

```

We see the largest concentration of values are at the start and end of the histogram, at 0-20 years and 60-73 years. Looking at the potential predictors related to Tenure, the strongest relationship occurs between MonthlyCharges. Although the only two other continuous variables are MonthlyCharges and TotalCharges, MonthlyCharges has a relatively high correlation (.826) while TotalCharges is smaller (.248).The data also appears to be right skewed due to the concentration on the left of the histogram. We can see a large number of values around 73+ due to truncation of the tenure variable or perhaps because of unaccounted for noise. The large number at 0 is likely due to customers staying for less than a year.

The Churn variable is binary and thus cannot be made into a histogram. Based on the Churn bar chart, far more customers have not churned than have churned.

Column {data-width=500}
-----------------------------------------------------------------------

### Transform Variables

```{r, cache=TRUE}
ggpairs(select(telcodf,Churn,tenure,SeniorCitizen,Partner,Dependents,PhoneService))

ggpairs(select(telcodf,Churn,tenure,MultipleLines,InternetService,OnlineSecurity,OnlineBackup,DeviceProtection))

ggpairs(select(telcodf,Churn,tenure,TechSupport,StreamingTV,StreamingMovies,PaperlessBilling,PaymentMethod))


```
Churn Analysis {data-orientation=rows}
=======================================================================

Row
-----------------------------------------------------------------------

### Predict Customer Churn (Yes(0)/No(1))
For this analysis we will use a Linear Regression Model.

```{r, include=FALSE, cache=TRUE}
#the include=FALSE hides the output - remove to see
Churn_lm <- lm(Churn ~ . ,data = telcodf)
summary(Churn_lm)
```

```{r, include=FALSE, cache=TRUE}
#the include=FALSE hides the output - remove to see
tidy(Churn_lm)
```

### Adjusted R-Squared

```{r, cache=TRUE}
ARSq<-round(summary(Churn_lm)$adj.r.squared,2)
valueBox(paste(ARSq*100,'%'), icon = "fa-thumbs-up")
```

### RMSE

```{r, cache=TRUE}
Sig<-round(summary(Churn_lm)$sigma,2)
valueBox(Sig, icon = "fa-thumbs-up")
```

Row
-----------------------------------------------------------------------

### Regression Output

```{r,include=FALSE, cache=TRUE}
#knitr::kable(summary(MEDV_lm)$coef, digits = 3) #pretty table output
summary(Churn_lm)$coef
```

```{r, cache=TRUE}
# this version sorts the p-values (it is using an index to reorder the coefficients)
idx <- order(coef(summary(Churn_lm))[,4])  
out <- coef(summary(Churn_lm))[idx,] 
knitr::kable(out, digits = 3) #pretty table output
```

### Residual Assumptions Explorations

```{r, cache=TRUE}
plot(Churn_lm, which=c(1,2)) #which tells which plots to show (1-6 different plots)
```

Row
-----------------------------------------------------------------------

### Analysis Summary
After examining this model, we determine that there are some predictors that are not important in predicting customer churn, so a pruned version of the model is created by removing predictors that are not significant.

Row
-----------------------------------------------------------------------

### Predict Customer Churn Final Version
For this analysis we will use a pruned Linear Regression Model. The variables we removed include DeviceProtection (if customer has Device Protection), OnlineSecurity (If the customer has Online Security), StreamingTV (If they have Streaming TV support), PhoneService (if customer has phone service), Partner (if the customer has a partner), and Gender (Male or Female).

```{r, include=FALSE, cache=TRUE}
#the include=FALSE hides the output - remove to see
Churn_lm <- lm(Churn ~ . -DeviceProtection -OnlineSecurity -StreamingMovies -StreamingTV -PhoneService -Partner -gender,data = telcodf)
summary(Churn_lm)
```

```{r, include=FALSE, cache=TRUE}
#the include=FALSE hides the output - remove to see
tidy(Churn_lm)
```

### Adjusted R-Squared

```{r, cache=TRUE}
ARSq<-round(summary(Churn_lm)$adj.r.squared,2)
valueBox(paste(ARSq*100,'%'), icon = "fa-thumbs-up")
```

### RMSE

```{r, cache=TRUE}
Sig<-round(summary(Churn_lm)$sigma,2)
valueBox(Sig, icon = "fa-thumbs-up")
```


Row
-----------------------------------------------------------------------

### Regression Output

```{r, include=FALSE, cache=TRUE}
knitr::kable(summary(Churn_lm)$coef, digits = 3) #pretty table output
```

```{r, cache=TRUE}
# this version sorts the p-values (it is using an index to reorder the coefficients)
idx <- order(coef(summary(Churn_lm))[,4])  
out <- coef(summary(Churn_lm))[idx,] 
knitr::kable(out, digits = 3) #pretty table output
```

### Residual Assumptions Explorations

```{r, cache=TRUE}
plot(Churn_lm, which=c(1,2)) #which tells which plots to show (1-6 different plots)
```

Row
-----------------------------------------------------------------------

### Analysis Summary
After examining this model, looking at the residual plots we can see that the data is not perfect and there are some problems. There are some high values at the right of the Q-Q plot that may be due to the truncated nature of the data. The deviations from the line represent departures from normality. The curved nature of the Q-Q plot may suggest that there is less variance than expected.

For the Residuals vs Fitted plot, we can see two very distinct lines which means that there is a problem with the data set. Ideally, the plot would show the residuals randomly scattered to indicate a consistent and unbiased fit but we do not see that here. 

Reducing the predictors that did not help with prediction of customer churn actually had a negative impact on our fit statistics (R-square and RMSE (root mean squared error)) as they both slightly decreased by 1%.

From the following table, we can see the effect on Customer Churn by the predictor variables.

```{r, cache=TRUE}
#create table summary of predictor changes
predchang = data.frame(
  Variable = c('ContractOne year', 'TotalCharges', 'MonthlyCharges', 'MultipleLines', 'TechSupport','PaymentMethodElectronic check', 'InternetServiceDSL', 'PaperlessBilling','ContractTwo year', 'tenure', 'SeniorCitizen', 'OnlineBackup','InternetServiceFiber optic', 'Dependents', 'PaymentMethodMailed check','PaymentMethodCredit card (automatic)'),
  Direction = c('Decrease', 'Increase', 'Increase', 'Increase', 'Decrease','Increase', 'Decrease', 'Increase', 'Decrease','Decrease','Increase', 'Decrease', 'Decrease', 'Decrease', 'Decrease','Decrease')
)
knitr::kable(predchang) #pretty table output

```

Tenure Analysis {data-orientation=rows}
=======================================================================

Row {data-height=900}
-----------------------------------------------------------------------

### Predict Customer Tenure
![](Tenure-Prediction.jpg)


Conclusion 1
=======================================================================
### Summary

In Conclusion, we can see that our predictors do help to predict a customers tenure, with an r-squared value of .87. The most significant predictors for tenure are TotalCharges, MonthlyCharges, Contract, and PaymentMethod.

From this analysis, we can see that as these variables increase they:
```{r}
#final table summary of predictor changes
predtenurefnl = data_frame(Decrease_Tenure = 
                            c("If the customer has Multiple Lines",
                             "Whether the customer has churned (Churn)",
                            "If they are a Senior Citizen (SeniorCitizen)",
                            "N/A", "N/A"
                            ),
                    Increase_Tenure = c("Customers Total Charges",
                                          "Customer Monthly Charges",
                                        "Length of Contract",
                                        "Customers Payment Method",
                                        "If the customer has a partner (Partner)"
                                ))  
knitr::kable(predtenurefnl) #pretty table output
```

Additional Churn Analysis 1 {data-orientation=rows}
=======================================================================

Row {data-height=900}
-----------------------------------------------------------------------

### Predict Churn - Logistic Regression
![](Churn-NominalLogisticV2.jpg)

Conclusion 2
=======================================================================
### Summary

In Conclusion, we can see that our predictors do help to predict whether a customer will churn, with Tenure, InternetService, and Contract being the most significant variables.

Combining the results of both types of predictor models and only reporting where agreement was found, we can see that as these variables increase they:
```{r}
#final table summary of predictor changes
predchurnfnl = data_frame(Decrease_Prob_to_Churn = 
                            c("Customer Tenure",
                            "Whether the customer has online security (OnlineSecurity)",
                            "Whether the customer is a senior citizen (SeniorCitizen)",
                            "Whether the customer has internet service (InternetService)",
                            "Whether the customer has multple phone lines (MultipleLines)"
                            ),
                    Increase_Prob_to_Churn = c("Customers Total Charges",
                            "Customers Payment Method",
                            "The length of customer contract (Contract)",
                            "Whether the customer has paperless billing (PaperlessBilling)",
                            "N/A"
                            ))
knitr::kable(predchurnfnl) #pretty table output
```

Additional Churn Analysis 2 {data-orientation=rows}
=======================================================================

Row {data-height=900}
-----------------------------------------------------------------------

### Predict Churn - Decision Tree
![](Churn-DecisionTree.jpg)